Add prefetch header for data prefetching #34498

ijjk · 2022-02-17T20:44:49Z

This ensures we don't un-necessarily revalidate paths when they are being prefetched but not being navigated to as this can cause a lot of extra requests which currently requires disabling automatic prefetching to workaround.

Fixes: #17758
x-ref: #20521
x-ref: slack thread

# Conflicts: # packages/next/server/response-cache.ts

ijjk · 2022-03-02T23:32:20Z

Stats from current PR

Default Build (Increase detected ⚠️)

General Overall increase ⚠️

	vercel/next.js canary	ijjk/next.js add/prefetch-header	Change
buildDuration	16.1s	16.2s	⚠️ +116ms
buildDurationCached	6.4s	6.2s	-132ms
nodeModulesSize	372 MB	372 MB	⚠️ +598 B

Page Load Tests Overall decrease ⚠️

	vercel/next.js canary	ijjk/next.js add/prefetch-header	Change
/ failed reqs	0	0	✓
/ total time (seconds)	3.138	3.194	⚠️ +0.06
/ avg req/sec	796.56	782.7	⚠️ -13.86
/error-in-render failed reqs	0	0	✓
/error-in-render total time (seconds)	1.37	1.396	⚠️ +0.03
/error-in-render avg req/sec	1825.08	1791	⚠️ -34.08

Client Bundles (main, webpack) Overall increase ⚠️

	vercel/next.js canary	ijjk/next.js add/prefetch-header	Change
925.HASH.js gzip	179 B	179 B	✓
framework-HASH.js gzip	42 kB	42 kB	✓
main-HASH.js gzip	27.9 kB	27.9 kB	⚠️ +31 B
webpack-HASH.js gzip	1.44 kB	1.44 kB	✓
Overall change	71.5 kB	71.6 kB	⚠️ +31 B

Legacy Client Bundles (polyfills)

	vercel/next.js canary	ijjk/next.js add/prefetch-header	Change
polyfills-HASH.js gzip	31 kB	31 kB	✓
Overall change	31 kB	31 kB	✓

Client Pages

	vercel/next.js canary	ijjk/next.js add/prefetch-header	Change
_app-HASH.js gzip	1.36 kB	1.36 kB	✓
_error-HASH.js gzip	192 B	192 B	✓
amp-HASH.js gzip	309 B	309 B	✓
css-HASH.js gzip	327 B	327 B	✓
dynamic-HASH.js gzip	2.57 kB	2.57 kB	✓
head-HASH.js gzip	351 B	351 B	✓
hooks-HASH.js gzip	920 B	920 B	✓
image-HASH.js gzip	5.06 kB	5.06 kB	✓
index-HASH.js gzip	263 B	263 B	✓
link-HASH.js gzip	2.26 kB	2.26 kB	✓
routerDirect..HASH.js gzip	320 B	320 B	✓
script-HASH.js gzip	387 B	387 B	✓
withRouter-HASH.js gzip	319 B	319 B	✓
85e02e95b279..7e3.css gzip	107 B	107 B	✓
Overall change	14.8 kB	14.8 kB	✓

Client Build Manifests

	vercel/next.js canary	ijjk/next.js add/prefetch-header	Change
_buildManifest.js gzip	460 B	460 B	✓
Overall change	460 B	460 B	✓

Rendered Page Sizes

	vercel/next.js canary	ijjk/next.js add/prefetch-header	Change
index.html gzip	532 B	531 B	-1 B
link.html gzip	545 B	545 B	✓
withRouter.html gzip	525 B	526 B	⚠️ +1 B
Overall change	1.6 kB	1.6 kB	✓

Diffs

Diff for main-HASH.js

@@ -4632,7 +4632,12 @@
           // > receiving cookies, always supply the `credentials: 'same-origin'`
           // > option instead of relying on the default.
           // https://github.com/github/fetch#caveats
-          credentials: "same-origin"
+          credentials: "same-origin",
+          headers: opts.isPrefetch
+            ? {
+                "x-nextjs-prefetch": "1"
+              }
+            : {}
         }).then(function(res) {
           if (!res.ok) {
             if (attempts > 1 && res.status >= 500) {
@@ -4658,7 +4663,8 @@
         isServerRender,
         text,
         inflightCache,
-        persistCache
+        persistCache,
+        isPrefetch
       ) {
         var ref = new URL(dataHref, window.location.href),
           cacheKey = ref.href;
@@ -4669,7 +4675,8 @@
           dataHref,
           isServerRender ? 3 : 1,
           {
-            text: text
+            text: text,
+            isPrefetch: isPrefetch
           }
         )
           .catch(function(err) {
@@ -5933,7 +5940,8 @@
                                     _this.isSsr,
                                     false,
                                     __N_SSG ? _this.sdc : _this.sdr,
-                                    !!__N_SSG && !isPreview
+                                    !!__N_SSG && !isPreview,
+                                    false
                                   )
                                 : _this.getInitialProps(Component, {
                                     pathname: pathname,
@@ -6223,6 +6231,7 @@
                                       false,
                                       false,
                                       _this.sdc,
+                                      true,
                                       true
                                     )
                                   : false;
@@ -6322,14 +6331,19 @@
             key: "_getFlightData",
             value: function _getFlightData(dataHref) {
               // Do not cache RSC flight response since it's not a static resource
-              return fetchNextData(dataHref, true, true, this.sdc, false).then(
-                function(serialized) {
-                  return {
-                    fresh: true,
-                    data: serialized
-                  };
-                }
-              );
+              return fetchNextData(
+                dataHref,
+                true,
+                true,
+                this.sdc,
+                false,
+                false
+              ).then(function(serialized) {
+                return {
+                  fresh: true,
+                  data: serialized
+                };
+              });
             }
           },
           {

Diff for index.html

@@ -19,7 +19,7 @@
       defer=""
     ></script>
     <script
-      src="/_next/static/chunks/main-d899d849ecf228ef.js"
+      src="/_next/static/chunks/main-25873de3a23095c3.js"
       defer=""
     ></script>
     <script

Diff for link.html

@@ -19,7 +19,7 @@
       defer=""
     ></script>
     <script
-      src="/_next/static/chunks/main-d899d849ecf228ef.js"
+      src="/_next/static/chunks/main-25873de3a23095c3.js"
       defer=""
     ></script>
     <script

Diff for withRouter.html

@@ -19,7 +19,7 @@
       defer=""
     ></script>
     <script
-      src="/_next/static/chunks/main-d899d849ecf228ef.js"
+      src="/_next/static/chunks/main-25873de3a23095c3.js"
       defer=""
     ></script>
     <script

Default Build with SWC (Increase detected ⚠️)

General Overall increase ⚠️

	vercel/next.js canary	ijjk/next.js add/prefetch-header	Change
buildDuration	19.2s	19.3s	⚠️ +29ms
buildDurationCached	6.3s	6.3s	⚠️ +8ms
nodeModulesSize	372 MB	372 MB	⚠️ +598 B

Page Load Tests Overall decrease ⚠️

	vercel/next.js canary	ijjk/next.js add/prefetch-header	Change
/ failed reqs	0	0	✓
/ total time (seconds)	3.19	3.175	-0.02
/ avg req/sec	783.81	787.51	+3.7
/error-in-render failed reqs	0	0	✓
/error-in-render total time (seconds)	1.405	1.428	⚠️ +0.02
/error-in-render avg req/sec	1779.18	1750.16	⚠️ -29.02

Client Bundles (main, webpack) Overall increase ⚠️

	vercel/next.js canary	ijjk/next.js add/prefetch-header	Change
925.HASH.js gzip	178 B	178 B	✓
framework-HASH.js gzip	42.3 kB	42.3 kB	✓
main-HASH.js gzip	28.1 kB	28.2 kB	⚠️ +43 B
webpack-HASH.js gzip	1.45 kB	1.45 kB	✓
Overall change	72.1 kB	72.1 kB	⚠️ +43 B

Legacy Client Bundles (polyfills)

	vercel/next.js canary	ijjk/next.js add/prefetch-header	Change
polyfills-HASH.js gzip	31 kB	31 kB	✓
Overall change	31 kB	31 kB	✓

Client Pages

	vercel/next.js canary	ijjk/next.js add/prefetch-header	Change
_app-HASH.js gzip	1.36 kB	1.36 kB	✓
_error-HASH.js gzip	179 B	179 B	✓
amp-HASH.js gzip	313 B	313 B	✓
css-HASH.js gzip	324 B	324 B	✓
dynamic-HASH.js gzip	2.56 kB	2.56 kB	✓
head-HASH.js gzip	351 B	351 B	✓
hooks-HASH.js gzip	921 B	921 B	✓
image-HASH.js gzip	5.2 kB	5.2 kB	✓
index-HASH.js gzip	261 B	261 B	✓
link-HASH.js gzip	2.33 kB	2.33 kB	✓
routerDirect..HASH.js gzip	322 B	322 B	✓
script-HASH.js gzip	388 B	388 B	✓
withRouter-HASH.js gzip	317 B	317 B	✓
85e02e95b279..7e3.css gzip	107 B	107 B	✓
Overall change	14.9 kB	14.9 kB	✓

Client Build Manifests

	vercel/next.js canary	ijjk/next.js add/prefetch-header	Change
_buildManifest.js gzip	458 B	458 B	✓
Overall change	458 B	458 B	✓

Rendered Page Sizes Overall increase ⚠️

	vercel/next.js canary	ijjk/next.js add/prefetch-header	Change
index.html gzip	530 B	531 B	⚠️ +1 B
link.html gzip	543 B	543 B	✓
withRouter.html gzip	525 B	525 B	✓
Overall change	1.6 kB	1.6 kB	⚠️ +1 B

Diffs

Diff for main-HASH.js

@@ -4632,7 +4632,12 @@
           // > receiving cookies, always supply the `credentials: 'same-origin'`
           // > option instead of relying on the default.
           // https://github.com/github/fetch#caveats
-          credentials: "same-origin"
+          credentials: "same-origin",
+          headers: opts.isPrefetch
+            ? {
+                "x-nextjs-prefetch": "1"
+              }
+            : {}
         }).then(function(res) {
           if (!res.ok) {
             if (attempts > 1 && res.status >= 500) {
@@ -4658,7 +4663,8 @@
         isServerRender,
         text,
         inflightCache,
-        persistCache
+        persistCache,
+        isPrefetch
       ) {
         var ref = new URL(dataHref, window.location.href),
           cacheKey = ref.href;
@@ -4669,7 +4675,8 @@
           dataHref,
           isServerRender ? 3 : 1,
           {
-            text: text
+            text: text,
+            isPrefetch: isPrefetch
           }
         )
           .catch(function(err) {
@@ -5933,7 +5940,8 @@
                                     _this.isSsr,
                                     false,
                                     __N_SSG ? _this.sdc : _this.sdr,
-                                    !!__N_SSG && !isPreview
+                                    !!__N_SSG && !isPreview,
+                                    false
                                   )
                                 : _this.getInitialProps(Component, {
                                     pathname: pathname,
@@ -6223,6 +6231,7 @@
                                       false,
                                       false,
                                       _this.sdc,
+                                      true,
                                       true
                                     )
                                   : false;
@@ -6322,14 +6331,19 @@
             key: "_getFlightData",
             value: function _getFlightData(dataHref) {
               // Do not cache RSC flight response since it's not a static resource
-              return fetchNextData(dataHref, true, true, this.sdc, false).then(
-                function(serialized) {
-                  return {
-                    fresh: true,
-                    data: serialized
-                  };
-                }
-              );
+              return fetchNextData(
+                dataHref,
+                true,
+                true,
+                this.sdc,
+                false,
+                false
+              ).then(function(serialized) {
+                return {
+                  fresh: true,
+                  data: serialized
+                };
+              });
             }
           },
           {

Diff for index.html

@@ -19,7 +19,7 @@
       defer=""
     ></script>
     <script
-      src="/_next/static/chunks/main-d899d849ecf228ef.js"
+      src="/_next/static/chunks/main-25873de3a23095c3.js"
       defer=""
     ></script>
     <script

Diff for link.html

@@ -19,7 +19,7 @@
       defer=""
     ></script>
     <script
-      src="/_next/static/chunks/main-d899d849ecf228ef.js"
+      src="/_next/static/chunks/main-25873de3a23095c3.js"
       defer=""
     ></script>
     <script

Diff for withRouter.html

@@ -19,7 +19,7 @@
       defer=""
     ></script>
     <script
-      src="/_next/static/chunks/main-d899d849ecf228ef.js"
+      src="/_next/static/chunks/main-25873de3a23095c3.js"
       defer=""
     ></script>
     <script

Commit: 8010947

Timer

I'm not sure this change should be made without a more formal RFC process first. I could see it argued both ways.

ijjk · 2022-05-22T18:05:38Z

Closing in favor of fresh PR

Continuation of #34498 this updates to send a `purpose: prefetch` header (related [w3c discussion](w3c/resource-hints#74)) when prefetching data routes and then on the server we skip revalidating when this header is set. When a client-transition is actually made we send a background HEAD (non-blocking) request to the data route without the `purpose: prefetch` header which signals a revalidation should occur if the data is stale. This helps alleviate the number of revalidations occurring currently caused by prefetches as a path can be prefetched but not visited during a session so may not need to be revalidated yet. Fixes: #17758 x-ref: #20521 x-ref: [slack thread](https://vercel.slack.com/archives/C031QM96T0A/p1645129445819139?thread_ts=1645124478.067049&cid=C031QM96T0A)

Add prefetch header for data prefetching

b2c85ad

ijjk added created-by: Next.js team PRs by the Next.js team. type: next labels Feb 17, 2022

This comment has been minimized.

Sign in to view

ijjk marked this pull request as ready for review February 17, 2022 21:17

ijjk requested review from timneutkens, shuding and huozhi as code owners February 17, 2022 21:17

ijjk marked this pull request as draft February 17, 2022 21:17

ijjk added 2 commits February 25, 2022 19:03

Merge branch 'canary' into add/prefetch-header

79de551

Merge branch 'canary' into add/prefetch-header

8010947

# Conflicts: # packages/next/server/response-cache.ts

ijjk marked this pull request as ready for review March 2, 2022 23:31

ijjk requested a review from Timer March 2, 2022 23:31

Timer reviewed Mar 15, 2022

View reviewed changes

ijjk closed this May 22, 2022

ijjk deleted the add/prefetch-header branch May 22, 2022 18:05

ijjk mentioned this pull request May 26, 2022

Update to not trigger revalidation during prefetch #37201

Merged

github-actions bot locked as resolved and limited conversation to collaborators Jun 22, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add prefetch header for data prefetching #34498

Add prefetch header for data prefetching #34498

ijjk commented Feb 17, 2022 •

edited

Loading

This comment has been minimized.

ijjk commented Mar 2, 2022

Diffs

Diffs

Timer left a comment

ijjk commented May 22, 2022

Add prefetch header for data prefetching #34498

Add prefetch header for data prefetching #34498

Conversation

ijjk commented Feb 17, 2022 • edited Loading

This comment has been minimized.

ijjk commented Mar 2, 2022

Stats from current PR

Diffs

Diffs

Timer left a comment

Choose a reason for hiding this comment

ijjk commented May 22, 2022

ijjk commented Feb 17, 2022 •

edited

Loading